Impact of Local Data Characteristics on Learning Rules from Imbalanced Data

In this paper we discus improving rule based classifiers learned from class imbalanced data. Standard learning methods often do not work properly with imbalanced data as they are biased to focus on the majority classes while " disregarding " examples from the minority class. The class imbalance affects various types of classifiers, including the rule-based ones. These difficulties include two groups of reasons – algorithmic and data level ones. The algorithmic factors include the following issues. First, most algorithms induce rules using the top-down technique, which hinders finding rules for smaller sets of learning examples, especially from the minority class. It is also connected with using improper evaluation measures to guide the search for best conditions in the induced rule and also for further rule pruning. Secondly, most algorithms use a greedy sequential covering approach, which may increase the data fragmentation and results in weaker rules, i.e., supported by a small number of learning examples. The " weakness " of the minority class rules influences classification strategies, where minority rules have a smaller chance to contribute to the final classification decision. The other difficulties concern characteristics of imbalanced data distributions. Learning classifiers becomes particularly difficult when other data characteristics occur together with imbalanced distribution of classes, such as decomposition of the minority class into many rare sub-concepts, too extensive overlapping of decision classes or presence of minority class examples inside the majority class regions. In our previous study [2], these data difficulty factors have been associated with different types of examples from the minority class: safe (located in the homogeneous regions populated by the examples from one class only), borderline, rare cases and outliers. The aim of this study is to present two different, recent algorithms proposed by K.Napierla and J.Stefanowski for inducing classification rules from imbal-anced data and to show the usefulness of studying local data characteristics for two sub-tasks: (1) improving rule classifiers by incorporating types of examples into the induction strategy; (2) applying the analysis of minority class examples to identify differences in performance of these algorithms and establishing their area of competence. The BRACID (Bottom-up induction of Rules And Cases for Imbalanced Data) algorithm is constructed following the critical analysis of limitations of current rule algorithms [3]. Its main features include: the bottom-up generalization of the most specific rules representing single examples in order to overcome the problems of data fragmentation; using twofold rule-based and instance-based